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© An image processing device and method for 
identifying an input image and a copier including 
such an image processing device and method. The 
image processing device (1, 2, 3) forms a gradated 
image from the input image, determines whether the 
gradated image includes a specified pattern, extracts 
a portion of the input image corresponding to the 
specified pattern, and compares the extracted por- 
tion of the input image with a reference pattern to 
determine a degree of resemblance between the 
extracted portion of the input image and the refer- 
ence pattern. 
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BACKGROUND OF THE INVENTION 

Field Of The Invention 

This invention relates to an image processing 5 
device and method and, in particular, to an image 
processing device and method for preventing the 
counterfeiting of bank notes, negotiable securities, 
and the like. The invention also relates to a copier 
including such an image processing device and to 
method. 

Description of the Related Art 



The development in recent years of copying 
devices such as full-color copiers has enhanced 
the quality of reproduction to a level at which one 
cannot distinguish the copy from the original with 
the naked eye. Such faithful reproductions can now 
be obtained inexpensively. However, we must con- 
sider that along with this benefit comes the in- 
creased risk of copiers being put to such illicit uses 
as counterfeiting money, negotiable securities, and 
other original documents whose reproduction is 
legally prohibited. To avert this danger, various 
devices have been developed to prevent copiers 
from being used for counterfeiting. One such de- 
vice is the image processing device disclosed in 
Patent Report 2-21 0481 . 

That image processing device reads the origi- 
nal image placed on the glass of the copier by 
scanning the entire original four times. It is used in 
full-color digital copy machines. Let us assume that 
a bank note has been loaded into the copy ma- 
chine. On the first scan, the device detects the 
rough location of the watermark which identifies 
paper money, the item which the machine is to be 
prevented from copying, on the assumption that 
the document on the glass is, in fact, a bank note. 
On the second scan, the device detects the exact 
position of the bill and the angle at which it is 
placed, i.e., the exact position coordinates of the 
bill on the glass. On the third scan, the device 
calculates the position coordinates of the red seal 
stamped on the bill from the exact position of the 
bill obtained in the second scan. Based on these 
position coordinates, it extracts an image from the 
region in which the seal is placed, and it makes a 
judgment as to whether there is in fact a red seal 
on this item. By scanning the item several times, 
the device is able to detect whether or not the 
document loaded in the copier is one which it 
would be illegal to copy. If the device detects that 
a prohibited document such as a bank note has 
been loaded to be copied, it will perform its speci- 
fied anti-counterfeiting operations on the fourth 
scan, such as making the entire display go black 
and preventing the copier from operating. 



Since existing devices such as that described 
above require a number of scans to detect a bank 
note or other item which cannot be copied, arriving 
at a judgment is time-consuming. Furthermore, not 
all full-color copiers use a four-scan copying 
scheme like the one described above. Some scan 
the original three times, and others only once. The 
processing device described above cannot be used 
with these types of copiers. Another problem is that 
an extremely large memory capacity is needed 
merely to store data concerning the size of paper 
money. Inevitably, then, only a limited number of 
kinds of money can be detected. 

J5 SUMMARY OF THE INVENTION 

In view of the difficulties discussed above, an 
objective of this invention is to provide an image 
processing device which is capable of detecting a 

20 bank note or other proscribed document by scan- 
ning it only once; which does not require a large 
memory capacity; which is capable of performing 
high-speed processing in real time; and which can 
be produced at a low cost. Another object of this 

26 invention is to provide a copier including such an 
image processing device. 

To achieve the objectives outlined above, the 
image processing device of this invention is pro- 
vided with a device to detect a pattern which 

30 resembles a specified pattern within image data 
which it has received; a device to extract this 
pattern, once detected, from the image data; and a 
device to calculate the degree of resemblance be- 
tween the extracted pattern and a reference pattern 

35 which was established beforehand. 

The device to detect a pattern has the ability to 
match a gradated pattern with the reference pattern 
which was established beforehand. The device to 
detect a pattern can comprise a device to match a 

40 gradated pattern with the reference pattern which 
was established beforehand and a device to select 
a pattern. This latter device outputs to a second 
pattern extraction device the results of the match- 
ing process performed by the matching device. 

45 These results comprise of the spatial relationships 
between the patterns detected by the matching 
device. 

The device to extract a pattern from the image 
data is capable of detecting certain specified loca- 

so tions in the pattern which has been detected, such 
as the apex of a corner or the center of a circle. 
The device to calculate the degree of resemblance 
has a number of items of fuzzy knowledge whose 
creation was based on each of the reference pat- 

55 terns which were established beforehand. The cal- 
culation device performs fuzzy inferences using the 
relevant fuzzy knowledge. 
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The device to detect a pattern, the device to 
extract a pattern, and the device to calculate the 
degree of resemblance all use a surface sequence 
scheme to process each data set received for each 
color component. The image processing device 
also has a device to select as recognized patterns 
those which the calculating device has determined 
to have a high resemblance to the reference and to 
store position data concerning these recognized 
patterns in the storage unit; and a device to judge 
whether or not each of the recognized patterns is in 
the correct spatial relationship with the others 
based on the position data for the recognized pat- 
terns which are extracted from each set of data for 
each color component and stored. In this way the 
device can judge whether or not the image data 
obtained during processing comprise the specified 
pattern to be detected. 

The device to calculate the degree of resem- 
blance has a number of reference patterns cor- 
responding to specified patterns which are found in 
various locations among the image data from which 
a single pattern is to be detected. The image 
processing device obtains the degree of resem- 
blance of the reference patterns to the pattern 
extracted by the extraction device. It labels as a 
recognized pattern any pattern whose degree of 
resemblance to the reference pattern exceeds a 
given reference value. Based on the spatial rela- 
tionship among the various recognized patterns 
which have been detected, it renders a judgment 
as to whether or not the image data being pro- 
cessed are those of the pattern to be detected. 

The image processing system according to this 
invention generally operates as follows. The sys- 
tem forms a gradated image with lower resolution 
than the image for which it has received data. It 
performs a rough search of this gradated image 
and attempts to match it to a reference pattern. In 
this way it detects patterns which are possible 
matches for the specified pattern which is to be 
detected. It then transmits to a storage unit the 
image data concerning these possible matches. 
Finally, it renders a judgment as to whether each 
possible match which was detected is or is not the 
pattern which was specified. This judgment is 
based on the data which are stored in the storage 
unit independently of the image data which were 
read out previously and on the results of the rough 
search. 

The copier according to this invention is 
equipped with any one of the image processing 
devices disclosed above. The image data output by 
a document reading device in the copier are input 
in parallel fashion to both the image processing 
device and a color signal converter. The image 
processing device renders a judgment as to wheth- 
er or not the object being copied is an object which 



was specified ahead of time, such as a bank note. 
If the object is one which was specified in advance, 
a control signal is sent to a specified processing 
device in the copier to prevent the copying opera- 

5 Bon from proceeding. 

In response to the image data which it re- 
ceives, the device to detect a specified pattern 
forms a gradated image, at lower resolution, of the 
pattern which it is to detect. It performs a rough 

ro search of the image data and attempts to match 
any patterns it finds to the gradated image. In this 
way it detects patterns which are possible matches 
for the specified patterns which are to be detected. 
It then transmits the image data concerning these 

75 possible matches to the device to extract a speci- 
fied pattern and stores them. Based on these 
stored image data, the device to extract a specified 
pattern extracts an image from a specified region 
independently of the processing performed during 

20 the rough search, and it transmits this image to a 
second device to match the specified pattern. This 
second device attempts to match this pattern and 
renders a judgment as to whether each possible 
match which was detected is or is not a pattern 

25 which was specified. Since the detection device 
extracts the rough candidates for matches, only a 
relatively small number of matching operations 
need be performed before the final pattern match- 
ing process is performed. And even though the 

30 image data are sent to the detection device one by 
one, the detection process can be performed with- 
out reference to the status of subsequent process- 
ing, i.e., without waiting for the results of that 
processing, because the data concerning the possi- 

35 ble candidates for matching have been stored tem- 
porarily in the device to extract data. This allows 
the processing to be performed in real time and at 
high speed. 

40 BRIEF DESCRIPTION OF THE D RAWINGS 

Figure 1 is a block diagram of the overall con- 
figuration of an embodiment of the image pro- 
cessing device of this invention; 

45 Figure 2 is a block diagram of the internal 
structure of the device to detect a specified 
pattern (the detection device); 
Figure 3 is a block diagram of the internal 
structure of the device to perform the averaging 

so operation; 

Figure 4 is a block diagram of the internal 
structure of the flip-flop groups in the detection 
device; 

Figure 5 is a block diagram of the internal 
55 structure of the decoder circuit in the detection 
device; 

Figure 6 is a diagram of the mask used by the 
search unit which looks for corners; 
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Figure 7 is a flow chart illustrating the workings 
of the search unit which looks for corners; 
Figure 8 is a diagram illustrating the basic ring 
used by the search unit which looks for marks; 
Figures 9(A)-9(B) are diagrams illustrating how 5 
the preliminary judgment unit works; 
Figure 10 is a block diagram of the internal 
structure of the device to extract a specified 
pattern and the device to match it against a 
reference pattern; w 
Figure 11 is a diagram illustrating how corner 
detection works in the position detection unit; 
Figure 12 is a flow chart illustrating the function 
of corner detection in the position detection unit; 
Figure 13 is a diagram showing how mark de- 75 
tection works in the position detection unit; 
Figure 14 is a flow chart illustrating the function 
of mark detection in the position detection unit; 
Figure 15 is a flow chart illustrating how the 
device to match a specified pattern against a 20 
reference pattern works; 

Figures 16(A)-16(B) are diagrams illustrating 
how the device to match a specified pattern 
against a reference pattern works; 
Figure 17 are diagrams illustrating examples of 25 
rules and membership functions stored in the 
rule and MF memories in the device to match a 
specified pattern against a reference pattern; 
Figures 18(A)-18(B) are diagrams illustrating the 
automatic creation of fuzzy knowledge to be 30 
used by the matching device; 
Figure 19 is a side, cut-away view of an exam- 
ple of a copy machine in accordance with this 
invention; 

Figure 20 is a block diagram of an example of a 35 
copy machine in accordance with this invention; 
Figure 21 is a block diagram of the internal 
structure of the device to extract a specified 
pattern and the device to match it against a 
reference pattern in another embodiment of the 40 
image processing device of this invention; 
Figures 22(A)-22(C) are diagrams illustrating 
how the device works; 

Figure 23 is a diagram illustrating how data is 
organized in the memory which stores inference 45 
results in the device; 

Figure 24 is a flow chart illustrating the judg- 
ment processing performed by the CPU in the 
device; 

Figure 25 is another flow chart of the judgment so 
processing performed by the CPU in the device; 
Figure 26 is a diagram illustrating how the de- 
vice works; 

Figure 27 is a flow chart illustrating the judg- 
ment processing performed by the control CPU 55 
of a third embodiment of the image processing 
device of this invention; 



Figures 28(A)-28(C) are diagrams illustrating 
how the device works; 

Figure 29 is another diagram illustrating how the 
device works; 

Figure 30 is a diagram showing a modification of 
the image processing device of this invention; 
Figure 31 is a diagram showing another modi- 
fication of the image processing device of this 
invention; 

Figure 32 is a diagram showing yet another 
modification of the image processing device of 
this invention; 

Figure 33 is a flow chart illustrating the judg- 
ment processing performed by the control CPU 
in the modification; and 

Figure 34 is a flow chart illustrating the judg- 
ment processing performed by the control CPU 
in another modification of the device. 

DETAILED DESCRIPTION OF THE INVENTION 

We shall next give a detailed explanation, with 
reference to the appended drawings, of a first 
embodiment of the image processing device and 
system and the copier in which these are used 
which are designed according to this invention. The 
processing device in this example is installed in a 
full-color copy machine. It is designed to detect 
paper money when someone attempts to copy it 
and prevent the copying process from proceeding 
to completion. First, as shown in Figure 1, the 
image data read by the copier's image sensor are 
input into device 1, whose purpose is to detect a 
specified pattern. As these image data are scanned 
by an image sensor such as a CCD, they are 
transmitted successively, region by region, to de- 
vice 1 . The actual data which are sent are full color 
data for each of the red (R), green (G) and blue (B) 
components, with resolution of 4O0DPI. 

Detection device 1 compresses the image data 
it has received to form a gradated image. Using 
masks of specified sizes, it checks the area of this 
image for patterns which are possible matches with 
each specified pattern which is to be detected, 
e.g., a mark, figure, etc. If it detects a possible 
candidate, it roughly specifies a reference position, 
e.g., the center for a circle, the apex of a corner for 
a rectangle, etc., for it and transmits the relevant 
data to device 2, the device to extract a specified 

Extraction device 2 detects these reference 
positions more precisely, based on the image data 
before they were compressed. These data include 
the potential matches detected during the rough 
search. Device 2 also creates patterns to be used 
for comparison in the pattern matching process to 
be performed later. Potential matches are stored 
temporarily in a specified storage area in device 2. 
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Extraction device 2 makes a preliminary judgment 
as to whether the extracted patterns resemble each 
specified pattern to be detected, based on the 
detailed, non-compressed image. If it finds a pat- 
tern similar to a specified pattern, it extracts the 
portion of it which surrounds the reference position 
and transmits this portion to device 3, where it will 
be matched against the specified pattern to be 
detected. 

Device 3 uses fuzzy pattern matching logic to 
determine the degree of fit between each extracted 
pattern and its reference pattern, i.e., the specified 
pattern to be detected, and it judges whether this 
pattern is, in fact, the one which was being sought. 
If it is, device 3 outputs a specified control signal. 

In this example, then, a rough search is made 
of the data read by the copier in its normal fashion. 
In this rough search, the gradated data are 
matched one by one against a pattern which was 
synthesized from the reference pattern, and poten- 
tial candidates for a match are extracted. The data 
concerning these potential matches are stored in a 
storage area, and they are matched against the 
reference pattern by means of fuzzy inferences 
when the specified processing sequence has been 
completed. Each potential match is judged as to 
whether it is, in fact, the pattern being sought. 
Matching device 3 uses for comparative purposes 
a detailed image which approximates the original 
image. Before an attempt is made to match can- 
didates to this detailed image, their number has 
been reduced by conducting a rough search at 
high speed. It should be appreciated that can- 
didates can be evaluated swiftly and accurately 
when judgments are based on gradated image 
data. All processing is performed at high speed in 
real time. There is no need to temporarily interrupt 
one process while another is being carried out. The 
image can continue to be read, for example, or a 
rough search can be performed, while judgments 
are being processed. 

In this section we shall discuss exemplary con- 
figurations of each device referred to above. Device 
1, which detects a specified pattern, is pictured as 
a block diagram in Figure 2. Here we see that the 
image data are sent to averaging unit 10, where a 
specified number of neighboring pixels in the rel- 
evant data are combined to form larger units. The 
density of the pixels in each unit is averaged to 
create a gradated image consisting, as it were, of 
rather larger pixels. 

Averaging unit 10 can comprise, for example, 
the components shown in the block diagram in 
Figure 3. Each output from latches 11a through 
1 1 d, which are connected in four stages that se- 
quentially output data synchronized by a shift 
clock, is input into adder 12a. Each pixel in each 
line of the image data is input, in sequential order, 



into latch 11a. Each pixel is represented as mul- 
tivalued data which express its density (or grada- 
tion of brightness). Thus in each of latches 11a 
through 11d, a specified number of bits is set 

5 which correspond to these multivalued data. 

The first adder, 12a, adds the four values 
which represent the densities of the pixels stored in 
latches 11a through 11d, which are the last four 
pixels to have been input, i.e., the last four pixels 

io on the path which is being scanned, and transmits 
the result to the second adder, 12b, through one of 
its input terminals. The sum of the densities of 
specified pixels in the last three lines, which were 
stored in line memory 13 (Figure 2) is input into 

J5 the other input terminal of adder 12b via AND 
circuit 14. The result of the addition performed by 
adder 12b is sent to both the line memory 13 and 
to divider 1 5. 

The output of adder 12b is written into line 

20 memory 13 each time four pixels have been ad- 
ded, in response to a control signal which is not 
pictured in the diagram. In other words, the first 
through fourth pixels on the line being processed 
are input simultaneously into adder 12a. Based on 

25 the result of the addition performed by adder 12a, 
further addition is performed by adder 12b, after 
which the first write-in of data into line memory 13 
is performed. When the fifth through eighth pixels 
have been added, the result is input into memory 

30 13 in the second write-in. This procedure is per- 
formed repeatedly. In this way, the result of the 
addition of every four pixels on line 1 is stored in 
order in line memory 13. 

When line 2 is processed, the result of adding 

35 every four pixels is obtained from latches 11a 
through 11d and adder 12a just as with line 1. At 
the same time, however, previous results of addi- 
tion which were stored in line memory 13 are read 
out, and adder 12b adds the results obtained by 

40 adding every four pixels on line 2 to the cor- 
responding results for every four pixels on line 1 . In 
this way the densities of eight pixels, four of which 
are contiguous to the other four in each of two 
rows, are added together. The result of this addi- 

45 tion is stored in line memory 13. 

When line 3 (or line 4) is processed, the totals 
of the densities of certain pixels in lines 1 and 2 (or 
lines 1 through 3), which were obtained by the 
addition performed during the processing of the 

so previous lines and stored in line memory 13, are 
added to the corresponding densities in the current 
line. When line 4 is processed, the result of the 
addition performed by adder 12b, i.e., the total of 
the densities of 4 x 4, or 16, pixels, is sent to 

55 divider 15, where it is divided by 16 to obtain an 
average value. The control signal which is input 
into AND circuit 15 goes to zero once line 4 has 
been processed. Readout of data from line mem- 
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ory 13 is prohibited, the cumulated value is reset, 
and the data stored in line memory 13 are cleared. 

It is contemplated that by increasing or de- 
creasing the number of latches, it is possible to 
increase or decrease the number of pixels whose 
density is averaged. Increasing the number of 
latches will produce an image with lower resolution. 
By varying the number of latches and choosing an 
appropriate timing for the input of the control signal 
to AND element 14, we can obtain an average 
density for N x M pixels. Which conditions are 
appropriate will depend on the image to be 
matched (or detected). 

The average density of each sixteen pixels is 
used to construct an image consisting of new, 
larger, pixel units. The data concerning this new 
image are sent to binary processor 16, where at a 
certain threshold density they are converted to 
binary values. The resulting binary image is stored 
temporarily in multiple-line memory 17. This gen- 
erated image is more blurry than the original im- 
age. Details of the original patterns in the image 
have been lost, and only rough shapes can be 
discerned. By setting an appropriate threshold for 
the conversion to binary, we can produce an image 
in which the interior of a given pattern, such as the 
corner or one of the markings on a bank note (or 
the side of the bill), will appear totally black while 
the background is totally white. Even if a mis- 
register or other printing error in the original image 
as read out and input results in a slight discrep- 
ancy between the specified pattern which is stored 
and the pattern which is being checked, the dis- 
crepancy will vanish when the image is gradated. 
This scheme allows the image to be detected sim- 
ply and quickly. 

The output from the binary processor 16 is 
sent to search unit 18, where the gradated image 
with low resolution (actually, the binary data) is 
searched for a pattern of a given shape. That is to 
say, a search is made for a specified pattern in a 
region of a given size, such as a mask with dimen- 
sions N x N. This search will be explained below. 

Search unit 18, shown in Figure 2, comprises 
flip-flop group 18a; circuit 18b, which receives the 
output from flip-flop group 18a, obtains the point 
coordinates of the hits which indicate the positions 
of patterns representing possible matches, and out- 
puts those coordinates; decoder circuit 18c, which 
specifies the orientations and other features of the 
patterns; and line memory 18d, in which are stored 
the data representing detected patterns which are 
possible matches. 

An example arrangement of flip-flop group 1 8a 
is shown in Figure 4. In this example, the area 
being processed consists of 3 pixels of scanning x 
3 pixels of feed, or 9 (3 x 3) flip-flops FF. The three 
pixels W10, W11 and W12, which are in line in the 



feed orientation, are taken from the image data 
stored in line memory 17, synchronized by means 
of the shift clock SCLK, and input to the first 
column of flip-flops, F00, F10 and F20. New pixel 

5 data are input to W10, W11 and W12 sequentially 
according to timing provided by the shift clock, and 
the data are transmitted to the next column of flip- 
flops. The pixel data which were input for the 
binary image, Q00, Q01, Q02, Q10, Q11, Q12, 

10 Q20, Q21 and Q22, are output through the output 
terminal Q on each flip-flop. For a black pixel, the 
output of the flip-flop will be "1 ". 

When all scanning data up to the end of one 
line have been input, the device shifts down one 

is line in the direction of feed and inputs data from 
the beginning of that line. Thus scanning the 3 x 3 
pixel area (the mask) which is the domain of the 
search is equivalent to moving over the surface of 
the document. 

20 Each of the outputs, Q0O, Q01, Q02, Q10, Q11, 

Q12, Q20, Q21 and Q22, is input into decoder 
circuit 18c. An example of decoder circuit 18c is 
pictured in Figure 5. If the pattern of 3 x 3 pixels 
which is output by flip-flop group 1 8a is one of the 

25 eight patterns (Hit 0 to Hit 7), the output of the 
corresponding AND element will go high. If the 
output from decoder circuit 18c for Hit 0 goes high, 
we know that there is a dark area in the top left- 
hand corner of the grid, centered around flip-flop 

30 FF11. If the output for Hit 4 goes high, we know 
that there is a dark area covering the upper part of 
the grid, centered around flip-flop FF11. In this 
example, the object of the search is a corner. 
Therefore, patterns other than these eight, such as 

35 when the central pixel is in the middle of a border 
or within or outside a dark area, will not be de- 
tected. 

Circuit 18b is connected to the addresses in 
multiple line memory 17. It outputs to flip-flop 

40 group 18a the addresses of images which must be 
input, and it outputs to line memory I8d the coordi- 
nates (XY) of the pixels output by FF1 1 (Figure 4), 
the central flip-flop in group 18a. The output of 
decoder circuit 1 8c and the central coordinates XY 

45 at that moment are stored in line memory 18d. The 
detection results for several lines worth of data are 
also stored in line memory 18d. In this example, 
nine flip-flops representing 3x3 pixels were used 
to detect the corner of a figure and the states of 

so the decoder circuit 18c were those shown in the 
diagram. However, the number of flip-flops used 
and the conditions for the decoder circuit 18c are 
preferably selected according to the shape of the 
pattern one wishes to detect and which portion of it 

55 one chooses to focus on. It would, of course, be 
possible to detect a circle or some other shape as 
required. 
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In the embodiment described above, the 
search for the specified pattern to be detected was 
carried out by hardware. It would also be possible 
to handle this processing via software, as will be 
discussed below. We shall next explain how a 
search is made for a corner, as in the embodiment 
discussed above, and how a search is made for a 
circle (or a mark). 

As shown in Figure 6, a 5 x 5 pixel mask is 
used, in which the object of the search is the 
central pixel. The first step is to number the pixels 
of the exterior and interior peripheries. For conve- 
nience, we begin with the leftmost pixel in the top 
row, which is assigned the address "0". The pixels 
in the exterior periphery are then numbered in 
ascending order, moving clockwise, and the pixels 
of the interior periphery are numbered in the same 
way, beginning with the leftmost pixel in the high- 
est row of interior pixels. Thus, an address i be- 
tween 0 and 7 is assigned to each interior periph- 
eral pixel, and an address i between 0 and 15 to 
each exterior peripheral pixel. If an address for a 
pixel falls outside of this range (0 to 7 for the 
interior periphery and 0 to 15 for the exterior pe- 
riphery) in each processing operation, the proper 
address can be obtained by adding or subtracting 
8 or 16, respectively. If, for example, i = 16 for an 
address on the exterior periphery, the actual ad- 
dress will be 0. If i = -1, the actual address will be 
15. 

The four corners of the mask are numbered 
starting from the upper left corner, which is called 
angle 0, in ascending order moving clockwise. The 
midpoints of the sides of the mask are numbered 
starting from the top, which is called angle 4, and 
moving clockwise, so that the other midpoints are 
labeled angles 5, 6 and 7. After the mask is num- 
bered in this way, the relevant pixel data in it are 
searched following the processing steps enumer- 
ated below to determine whether or not the speci- 
fied pattern to be detected can be found. 

First, the mask is scanned with reference to the 
low-resolution gradated image data which were re- 
ceived. If there is a black pixel in the center, we 
proceed to the next step, which is the search for a 
starting point. In this example, it has been estab- 
lished that this portion of the specified pattern (the 
object of the search) is black, i.e., a black pixel. 
However, with the opposite pattern, black and white 
would be inverted, and the central pixel would be 
white. The subsequent procedure would be the 
same.) 

A search is made of the periphery of the mask 
for the point where the black pixels commence, i.e., 
the point where the area changes from white to 
black pixels. In other words, the system resets at i 
= 0 and searches, incrementing each time by one, 
for a point where pixel number i is black and pixel 



number (i - 1) is white. If a starting point is not 
found when i = 16, the judgment is made that the 
pattern in this mask is not a possible candidate for 
a match, and the rough search is completed. If a 

5 starting point is found, we proceed to the next step, 
which is searching for an endpoint. In the example 
which is pictured, when we increment i until we 
reach i = 6, we find a black pixel (shown by 
hatching in the drawing). Since the previous pixel, 

jo pixel 5 (i - 1), is white, we conclude that i = 6 is 
the starting point of the black pixels. 

We now search for the point on the periphery 
of the mask where the black pixels end, i.e., th.8 
point where black pixels go to white pixels. Since 

75 in this example we are searching for a corner 
which is a right angle, we can predict where the 
endpoint ought to be with respect to the starting 
point if this is indeed the specified pattern. We 
jump to that point and judge whether or not it is a 

20 point at which black pixels end and white pixels 
commence. If it is not, then a judgment is made 
that the starting point detected in the previous 
processing is not the edge (or corner) of the speci- 
fied pattern. 

25 Accordingly, a judgment is made as to whether 
each of the following three conditions (1) - (3) 
below is met. If a condition is met, the system 
moves to the next step of angle detection process- 
ing. If any of the three conditions is not met, it is 

30 concluded, as described above, that the starting 
point which was detected is incorrect. The value i 
is augmented by 2, and we return to the second 
step of the starting point detection processing, in 
which we repeat the process of detecting the start- 

35 ing point for another location. If the corner does not 
form a right angle, the numerical value is changed 
in an appropriate fashion by adding the values 
given below to the value i, according to the angle. 

(1) If i is 3, 7, 11 or 15, i + 6 is a black pixel, 
40 and i + 7 is a white pixel, then i + 6 is the 

endpoint. 

(2) If i + 5 is a black and i + 6 a white pixel, 
then i + 5 is the endpoint. 

(3) If i + 4 is a black and i + 5 a white pixel, 
45 then i + 4 is the endpoint. 

In order to determine the orientation of a pat- 
tern which is a candidate for a match, we must 
determine which of the aforementioned angles 0 
through 7 it faces. To do this we cover the area 

so between the peripheral starting point and endpoint 
with black pixels, and cover the area between the 
endpoint + 1 and the starting point - 1 with white 
pixels. If one of angles 0 through 7 is among the 
black pixels (in this case, one of angles 0 through 

55 3), we say that the pattern is facing that angle. The 
number of that angle is used to indicate the ori- 
entation of the pattern. If several angles are found 
in the black pixels, the pattern is oriented toward 
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one of the four greater angles (angles 4 through 7). 
In actual practice, the angle can be obtained by 
performing the following calculations. 

Ki = (starting point + 3)/4 
K? = (endpoint)/4 

For both Ki and Kz, we discard fractional val- 
ues. If Ki is 4 or more, then Ki = K 2 - 4. If when 
we compare Ki and K2 we find that they are equal, 
then the value of Ki is used as the number of the 
angle the pattern faces (one of angles 0 through 3). 
If Ki and K2 are different, we add 4 to Ki to obtain 
the numerical value of the angle the pattern faces 
(one of angles 4 through 7). 

When a pattern is detected by the processing 
described above which looks like the specified 
corner, the coordinates of its hit point, i.e., the 
reference position for that corner (for the apex of a 
corner, the central position of the mask), are sent 
to preliminary judgment unit 19 along with the 
orientation of the angle. The flow of operations 
performed during a corner search is pictured in 
Figure 7. The corner which was searched for in this 
example need not be limited to the outer corner of 
a bank note. Any type of corner can be searched 
for, including a corner-like pattern on the interior 
portion of a bill or in any other document. 

A rough search is made by detecting a round 
pattern of a specified diameter in a specified area. 
More specifically, a standard ring of a specified 
diameter is used, as shown in Figure 8 (the plus 
indicates the center of the ring). A pattern which is 
found is matched against these rings; if it has a 
high degree of coincidence, it is judged to be a 
possible match. 

Line memory 18d in search unit 18, which is 
realized in hardware, stores a great many data 
which will not need to be sent to pattern extraction 
device 2 for the next step of processing. The data 
which are stored in line memory 18d are sent 
instead to preliminary judgment unit 19, where a 
judgment is rendered as to whether those data 
should be sent to and processed by pattern extrac- 
tion device 2 (preselection of hit points). 

In this example, a rough search is performed 
with reference to gradated data. This allows image 
data which potentially have the specified pattern to 
be identified swiftly and surely as possible match- 
es. At the same time, a large quantity of useless 
data are also detected. When the states of the 
pixels on the periphery lead to the judgment that a 
given pattern is not the specified pattern, that pat- 
tern can be eliminated without being output to 
pattern extraction device 2, obviating the need for 
subsequent more detailed judgments. Only the ac- 
tual hit points are output to memory 21 in pattern 
extraction device 2. 



To process this judgment, data stored in mem- 
ory I8d for which the output goes high (the hit 
points) are extracted, and effective hit points are 
detected based on the relative spatial relationships 

5 among these hit points and other hit points which 
are on their periphery. The position coordinates XY 
and the orientation data for these points, HIT 0 to 
7, are stored in memory 21 . 

To give one example, depending on the type of 

10 pattern being sought (its shape, size, etc.), if there 
is another hit point in the vicinity of a hit point 
which has been detected, or within a specified area 
around that hit point, then the first hit point will be 
regarded as valid. If a number of hit points are 

/5 found within a specified area, then an appropriate 
decision will be made, depending on the pattern, to 
consider only the periphery (Figure 9 (A)) or only 
the center (Figure 9 (B)). 

In the example discussed above, when a speci- 

20 tied number of pixels from the image data were 
averaged to create a gradated image, the resolu- 
tion was altered to a specified level in a single 
operation. It would alternatively be possible to alter 
the resolution gradually using a series of oper- 
as ations. 

Figure 10 is a block diagram illustrating an 
exemplary configuration of pattern extraction de- 
vice 2. As shown in the Figure 10, device 2 con- 
tinuously stores in memory 20 the RGB data (the 

30 variable density image) from detection device 1 
which have not yet been converted to binary val- 
ues. Memory 20 has a sufficient capacity to allow it 
to maintain storage of image data from the pixels 
(or the line) currently read by the image sensor to 

35 a specified earlier point (at least as large as the 
pattern to be detected). If its capacity is reached, 
the oldest data (the data acquired earliest) are 
erased and new data are written in their place. 
As was described earlier, the coordinate values 

40 and orientation of valid hit points output by prelimi- 
nary judgment unit 1 9 in detection device 1 (Figure 
2) are written into memory 21. The stored coordi- 
nate values are read out by control CPU 30. The 
CPU 30 sends various control signals to extraction 

45 device 2, thus controlling its operation. One such 
control signal is the command sent to memory 20 
telling it to output, based on the coordinate values 
read out from memory 21, the image data which 
contain the pattern associated with a valid hit point. 

50 Based on the coordinates read out by CPU 30, 

the corresponding image data stored in memory 20 
are sent to binary unit 22. Here the RGB image 
data are converted to binary values; however, the 
threshold value for this conversion need not be the 

55 same as that used by binary processor 16 in the 
pattern detection device 1 (Figure 2). Since the 
primary objective in device 1 was to detect the 
presence of possible matches to the specified pat- 
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tern, the threshold was set at a level which ba- 
sically rendered the interior of the pattern entirely 
dark. The threshold used by device 2 is set so as 
to extract more accurate hit points (specified posi- 
tions) and exhibit a detailed mosaic which reveals 
the characteristics of the detected pattern so that 
pattern recognition can be performed at a later 
stage. 

The binary data produced in this fashion are 
stored temporarily in detailed memory 24 and sent 
in order to position detection unit (CPU) 25. Here 
reference points are obtained for the matching op- 
eration to be performed in matching device 3. 
Based on data stored in program ROM 25a, which 
is connected to CPU 25, a certain specified posi- 
tion in the specified pattern to be detected is 
obtained. For a corner, this position is the accurate 
location of the apex, and for a mark, the accurate 
location of the center. 

In other words, position detection unit (CPU) 25 
operates in fundamentally the same way as prelimi- 
nary judgment unit 19, which was described above. 
It performs the processing which is appropriate for 
a pattern which is a corner or a mark, and it 
obtains a specified position within that pattern, 
which is the accurate location of the apex for a 
corner and the accurate location of the center for a 
mark. If when position detection unit (CPU) 25 
extracts this accurate position it judges that this 
candidate is not the specified pattern to be de- 
tected, output will cease as explained in more 
detail below with respect to Figures 11 and 12. 

In the rough search, the approximate apex of 
the corner (the hit point) is known, as well as the 
orientation of the corner. A hypothetical origin 0 is 
established in a specified position in the vicinity of 
hit point H and inside the corner. From this hy- 
pothetical origin 0, a horizontal X axis and a vertical 
Y axis are established (Steps 1 and 2). 

Next, the necessary constant is established for 
detection of a specified position in the pattern 
(Step 3). Moving along the X axis in steps of a 
specified width (indicated by arrows in Figure 11), 
we scan at each step in a positive (upward) and 
negative (downward) direction parallel to the Y axis, 
judging whether the pixels are black or white. We 
find the border between black and white (the bor- 
der where black ends and white begins will be 
called an edge hereafter) (Steps 4 and 5), and we 
draw a straight line following the edge which has 
been detected as closely as possible (segments 
indicated by hatching). The minimum mean square 
criterion, for example, might be used to cause the 
various edges which have been detected to ap- 
proximate a straight line (Steps 6 and 7). When the 
relevant processing is performed, the two edges 
which extend on both sides of the Y axis are 
regularized as straight lines L1 and L2. The inter- 



section of these two lines is found and labeled 
apex P (steps 8 and 9). 

The precise orientation of the corner can be 
computed from the slope of one of the straight 
5 lines (or edges) which have been detected or from 
the slope of the line bisecting the angle formed by 
the two straight lines (Step 10). The coordinates of 
the apex and the slope are transmitted to control 
CPU 30. 

10 If when the apex is extracted two edges cannot 
be found, or if an insufficient number of edge data 
are found, or if the angle formed by the two edge 
lines varies substantially from the specified angle, 
then a judgment will be rendered that the pattern 

75 found is not the corner we are seeking, i.e., it is not 
the specified pattern to be detected, and output will 
cease. 

As is illustrated in Figures 13 and 14, the hit 
point discovered in the rough search for a mark is 

20 considered to be the hypothetical center point O. 
Lines L3 through L6 are drawn in the four compass 
directions from center point O to form X and Y 
axes (Step 1 1 ). The necessary constants are estab- 
lished for the detection of specified positions in the 

26 pattern (Step 12). Since the radius of the mark is 
already known, the approximate location of its cir- 
cumference, i.e., its edge, can be determined. A 
line L' is drawn orthogonal to the radius, and this 
line is used to establish a window, i.e., an area, 

30 with a width a and a depth b (Steps 12 to 14). 

A histogram is drawn of the black pixels in the 
direction of the search (the radial direction) and the 
orthogonal direction. A certain height of the histo- 
gram Hth is chosen to be the threshold value. 

35 Levels greater than height Hth are considered to 
represent the mark as far as they continue. The 
point where they end is considered to be the start 
of the blank area. The border between these areas 
is considered to be the edge. Rather than making a 

40 judgment based solely on processing involving a 
threshold value, it would alternatively be possible to 
extract the border between the pattern and non- 
pattern portions based on the variation in the height 
of the histogram and its continuity (Steps 15 and 

45 16). 

The search for the edge is performed in a 
radial direction on both the left and right sides, and 
a center position P is obtained for the edges which 
have been extracted as a result of the search. In 

so the same way, the upper and lower edges are 
found, and center positions are obtained for them 
(Steps 17 and 18). 

When only one edge can be extracted, the 
radius of the target image can be used to deter- 

55 mine the center. This is done by moving back from 
the edge which has been detected along a radius a 
distance equal to the radius of the target and 
declaring that point to be the center. When the 
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center position and the radius have been deter- 
mined in this way, the relevant data are output to 
CPU 30 (Step 19). If a center cannot be found 
when a number of hypothetical center points have 
been detected as described above (because the 
diameters vary too much or edges cannot be de- 
tected on both sides of the diameter), a judgment 
will be rendered that the image data being pro- 
cessed do not contain the specified pattern to be 
detected, and output will cease. 

Since in this example only the data within the 
window are searched, the processing can be done 
at a high speed. If there is a specific shape within 
the specified pattern or a characteristic of some 
kind inside the mark, the data can be expressed as 
a four-directional histogram in the same way as 
was described above. Since the angle of rotation is 
known, it, too, can be transmitted to CPU 30 as 
needed. 

We shall next discuss the workings of pattern 
matching device 3. As is shown in Figure 10, the 
coordinate data for the apex and slope of a corner 
resembling the specified pattern or for the center 
position of a mark are sent from position detection 
unit (CPU) 25 to control CPU 30. Based on the 
data stored in program ROM 30a, control CPU 30 
specifies which of the image data stored in mem- 
ory 24 must be read out, and it decides which area 
of the image data should be matched against the 
specified pattern to be detected. The relevant im- 
age data in this region are transmitted from mem- 
ory 24 to fuzzy inference unit 35 based on a 
control signal from control CPU 30. Inferences are 
processed using the fuzzy knowledge stored in 
memories 36 and 37, which include rules and 
membership functions. A judgment is then ren- 
dered as to the degree of resemblance between 
the received image data and the previously estab- 
lished specified pattern to be detected (See Figure 
15). 

An example is shown in Figures 16(A)-16(B). 
Characteristic spaces are set up using pre-gra- 
dated image data. The number of characteristics is 
extracted by counting the actual number of set 
pixels in each characteristic space (each of the four 
regions shown in the example). In this way we find 
that characteristic space R1 has 24 pixels, space 
R2 has 26, space R3 has 30, and space R4 has 29. 

Using the characteristic counts which have 
been extracted, fuzzy inferences are performed 
according to previously established rules and 
membership functions (shown in Figure 17), and 
the degree of fit (i.e., the degree of resemblance) 
between these data and the reference pattern is 
determined and output. In the example shown, the 
degree of fit is 0.6. Since this is higher than 0.5, 
which is the judging standard, it is concluded that 
this image is identical to the reference pattern. 



In this example, the characteristic spaces are 
produced by drawing a lattice; however, the user 
may choose some other form of characteristic 
space which better suits his purposes, such as 

5 concentric circles or rectangles. The characteristic 
count which is extracted may also take other forms 
than the number of pixels per space. One could 
alternatively use the number of connected or dis- 
connected pixels, their orientation, or another char- 

io acteristjc. 

In this example, too, when the fuzzy knowledge 
was laid down, a number of items of fuzzy knowl- 
edge were established pertaining to a single speci- 
fied pattern to be detected. These various items of 

75 knowledge can all be invoked when inferences are 
processed. It might happen, for example, that the 
image data which were read are not identical to 
those of the specified pattern to be detected, but 
are spatially shifted by a certain amount due to a 

20 printing misregister or some other error. A frequent 
problem with marks is that the angle of rotation is 
unclear. To deal with such situations, we can hy- 
pothesize a misregister of the specified pattern to 
be detected or numerous possible angles of rota- 

25 tion when we are creating the fuzzy knowledge, 
and we can store these hypotheticals. 

An example concerning a corner is given in 
Figure 1 8(A). Map data are created by reading the 
target (or reference) pattern with a specified resolu- 

30 Hon, using an image whose central portion is cor- 
rectly registered as shown by solid lines in the 
drawing. These map data are equivalent to the 
image data output by binary unit 24 in pattern 
extraction device 2 at the time the specified pattern 

35 to be detected is read by the copier. In the same 
way, map data are created for an image which has 
been shifted horizontally, to the upper left (shown 
by dotted lines in the drawing), to the upper right, 
to the lower left, or to the lower right. 

40 For a mark, as shown in Figure 18(B), map 

data are created by reading the target (or refer- 
ence) pattern with a specified resolution, using an 
image whose angle of rotation is 0" (an image 
which is vertically oriented). These map data are 

45 equivalent to the image data output by binary unit 
24 in pattern extraction device 2 at the time the 
specified pattern to be detected is read by the 
copier. The polar coordinates of this image are 
changed so that their angles of rotation have been 

so shifted in 45-degree increments, and map data are 
created for the resulting images. For each of these 
images, an item of fuzzy knowledge is created. In 
this way numerous items of fuzzy knowledge are 
created by altering the angle of rotation of a single 

55 pattern (in this case, a mark). 

The numerous items of fuzzy knowledge 
achieved in this way are grouped into sets, all of 
which are used to draw inferences. The pattern 



10 



19 



EP 0 649 114 A2 



20 



with the highest inferred degree of fit is deemed to 
be the recognized specified pattern to be detected. 
This scheme allows us to extract likely matches 
and to process inferences concerning these pu- 
tative matches without regard for the direction in 
which a mark is rotated or the shifted position of a 
corner. 

It is also possible to create fuzzy knowledge 
without resorting to this type of grouping, by statis- 
tically processing every possible shifted image. In 
this case, there will ultimately be a single mass of 
fuzzy knowledge; however, when we create fuzzy 
knowledge by considering the direction of rotation 
or the horizontal shift of a single reference pattern, 
we are using the numerous items of knowledge 
mentioned in this invention. When we compare the 
two approaches, we find that the former, i.e., using 
sets of knowledge items, offers a higher rate of 
pattern recognition, while the latter offers a greater 
processing speed. The user must decide which 
choice is appropriate. 

If the image is shifted by one pixel (or by an 
integer multiple of one pixel), an addressing opera- 
tion can be performed. This consists of repeatedly 
performing an inference while shifting the reference 
pattern or the pattern one is trying to find to one 
side in a specified direction. This approach allows 
us access to the same operations and results as if 
we were using numerous items of knowledge, even 
though these items are not actually stored in the 
memory. It also allows an accurate judgment to be 
made even if the position of the pattern is shifted. 
The concept described herein of deriving numer- 
ous items of knowledge from a single pattern is 
broad enough to include both the actual storage of 
these items in a memory and the hypothetical use 
of these items described above. 

The degree of fit inferred on the basis of nu- 
merous items of knowledge as described above is 
output to a PPC (a copier) by way of control CPU 
30. When the degree of fit exceeds a given thresh- 
old value, the copier will conclude that the docu- 
ment presented is a bank note or some other item 
which cannot be legally copied. It will proceed to 
execute the program it has for prevention of forg- 
ery by, for example, prohibiting the copying opera- 
tion, outputting a totally black screen, etc. As an 
alternative, CPU 30 can render a judgment as to 
whether the item presented is a prohibited docu- 
ment, i.e., whether it contains one of the specified 
patterns to be detected, and it can output the 
result, e.g., a stop signal, to the copier. 

Figures 19 and 20 show an example in which 
the device is built into a copy machine. As shown 
in the Figure 19, the light emitted by lamp 52 is 
reflected by document 51 , which is loaded on glass 
50. The reflected light passes through optical sys- 
tem 53 and enters CCD 54, which is an image 



scanner. Here the image on the document is read. 
Lamp 52, as well as the flat mirror and other 
components which comprise optical system 53, 
moves at a specified speed as it scans the docu- 

5 ment. Line by line, CCD 54 reads the specified 
portion of document 51 , and it transmits the image 
data (R/G/B) to signal processing unit 55. 

Signal processing unit 55, which is shown in 
more detail in Figure 20, has an ordinary color 

to processing circuit 56 and image processing device 
57, which is the image processing device dis- 
cussed above as a feature of this invention. The 
image data is transmitted in parallel to color pro- 
cessing circuit 56 and to image processing device 

75 57. Color processing circuit 56 analyzes the colors 
into their various components, magenta (M), cyan 
(C), yellow (Y) and black (Bk), and outputs these 
data to printing device 58. The image is actually 
scanned four times. One of the four components 

20 (M, C, Y, or Bk) resulting from the first scan is 
output to laser driver 59, which is positioned on the 
input side of printing device 58. This causes a laser 
beam to be projected onto a specified location on 
light-sensitive drum 60. When the fourth scan has 

25 been completed, the copying operation is per- 
formed on the copy paper, and the finished copy 
61 is output. Since the actual mechanism which 
executes the copying operation is identical to that 
of existing copiers, we shall omit an explanation of 

ao its action. 

While the signal processing is being carried 
out by color processing circuit 56, image process- 
ing device 57 is performing its own processing in 
parallel to obtain the degree of resemblance be- 

35 tween the pattern it finds in the image data it is 
reading and the specified (or reference) pattern. If 
document 51, which is still being read, is a bank 
note or other document which may not be copied, 
the image processing device 57 will generate a 

40 control signal to halt the output of the laser driver 
59 or transmit a control signal to color processing 
circuit 56 to cause it to execute its various copy 
prevention routines for situations when copying is 
prohibited, such as making the entire finished copy 

45 61 go black. 

The judgment processing in this embodiment 
is performed in real time during the first scan. This 
device, then, can be used in copiers which employ 
a single-scan method. It can, needless to say, also 

so be used in copiers which employ a multiple-scan 
method. With copiers of this type, CPU 30 sends a 
control signal to pattern detection device 1, which 
is pictured in Figure 2. The detection process can 
be performed for a different specified pattern to be 

55 detected on each scan. 

In the example discussed above, a number of 
specified patterns can be detected on a single 
scan. Between the completion of the first scan and 
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the beginning of the second, a command can be 
transmitted from CPU 30 to detection device 1 
which changes all the parameters of the processing 
units (the threshold value for binary data and so 
on). By rewriting these parameters, we make it 5 
possible to detect a different group of specified 
patterns. This configuration, then, allows us to pre- 
vent the forgery of an extremely large number of 
prohibited documents, such as money, which serve 
as the specified pattern to be detected. Since only w 
a portion, and not the entire face, of a bill is 
detected and matched against a specified pattern, 
none of the requisite memories need have a large 
capacity. Though it can accurately prevent the 
forgery of a variety of documents, as discussed 75 
above, the device does not require an extraordinary 
overall memory capacity. 

Figure 21 shows a second embodiment of an 
image processing device designed according to 
this invention. In the first embodiment discussed 20 
previously, characteristics were extracted from the 
RGB data, which were transmitted pixel by pixel in 
a linear sequence, and the specified processing 
was then performed. In the second embodiment, 
which is suitable for copiers using a field-sequential 25 
scheme, all the R data, then all the G data, then all 
the B data of the image read for the printer are 
transmitted sequentially. 

We shall next give a brief explanation of how 
the second embodiment works. As shown in Figure 30 
1, pattern detection device 1 conducts a rough 
search, pattern extraction device 2 conducts a de- 
tailed search, and pattern matching device 3 pro- 
cesses a judgment with respect to the potential 
matches for the specified pattern which have been 35 
extracted. This much of the basic structure is the 
same as in the first embodiment. The second em- 
bodiment differs from the first in that the second 
embodiment constructs separate reference patterns 
to be used in the matching process for each of the 40 
sets of R, G and B data. A rough search and a 
detailed search are performed after each of the 
signals RGB, which are transmitted in field-sequen- 
tial fashion. In these searches, a characteristic 
count is extracted and potential matches are de- 45 
tected. Each time, an overall judgment is made as 
to the location of the characteristics, the degree of 
fit, and so on. A judgment is also made as to 
whether the image being processed is or is not 
identical to the specified pattern to be detected. 50 

If we compare the hardware of the second 
embodiment with that of the first embodiment, 
which is pictured in Figure 10, we see that the 
second embodiment has an additional memory 65 
for the temporary storage of the results of infer- 55 
encss performed after each signal is received. This 
memory is connected to CPU 30'. Outside of the 
fact that the items they process are the total pixels 



of R, G or B data received in field-sequential fash- 
ion, pattern detection device 1' and pattern extrac- 
tion device 2' have the same capabilities as their 
counterparts in the first embodiment. 

CPU 30' transmits data concerning potential 
matches which it receives from pattern extraction 
device 2' to fuzzy inference unit 35'. Based on a 
reference pattern represented by signals which cor- 
respond to its features (which are stored in rule 
memory 36' and MF memory 37'), unit 35' per- 
forms fuzzy inferences and obtains the degree of fit 
of each potential match. 

CPU 30' also judges whether each potential 
match should be considered an effective candidate 
(a candidate which should go into the pool to be 
judged as to whether it contains the specified pat- 
tern to be detected). The degree of fit and location, 
i.e., the position coordinates of the reference point 
obtained by position detection unit 25', of data 
concerning effective candidates are stored in mem- 
ory 65. As part of the judgment processing, the 
degree of fit obtained in the second embodiment is 
compared with a threshold value. Data concerning 
candidates whose degree of fit exceeds the pre- 
viously established threshold value, for example, 
70%, are extracted. 

An example is shown in Figures 22(A)-22(C). 
The R, G and B components of the image data are 
shown in Figures 22(A), 22(B) and 22(C), respec- 
tively. Pattern extraction device 2' extracts patterns 
(1) through (4) because of their resemblance to a 
specified pattern to be detected and sends them to 
CPU 30'. As a result of inferences performed by 
fuzzy inference unit 35', the degree of fit of pattern 
(1) with respect to the specified pattern to be 
detected, i.e., the reference pattern, is found to be 
85%, and that of patterns (2) through (4) is found to 
be 50%, 75% and 80%, respectively. 

Data concerning patterns (1), (3) and (4), 
whose degree of fit exceeds 70%, making them 
effective candidates for a match, are stored in 
memory 65. An example of how these data are 
organized to be stored is shown in Figure 23. 
Degree of fit is stored together with position, i.e., 
the coordinates of the reference position, and data 
concerning which specified pattern the candidate 
resembles are also stored, though they are not 
included in Figure 23. If there are a number of 
effective candidates among the data represented 
by a single signal, the data concerning each of 
them are stored as well, as indicated by the paren- 
theses in Figure 23. 

CPU 30' also judges whether the three infer- 
ence operations, one for R, one for G and one for 
B, have all been completed. When they have, CPU 
30' makes overall judgments of the data concern- 
ing the effective candidates stored in memory 65 
and renders a final judgment as to whether the 
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image being processed does or does not contain 
the specified pattern to be detected. It then outputs 
a specified control signal to the copier or another 
device (not shown). 

The actual steps in this process are shown in 
Figures 24 and 25. The desired signal, for exam- 
ple, the results of inferences concerning the R data 
(the position coordinates of effective candidates 
and so on), is read out (Step 21). A table which is 
stored in program ROM 30' is consulted to deter- 
mine the relationship between the specified pattern 
and the image which is to be detected, for exam- 
ple, the denomination of a bill. A determination is 
made as to what category the pattern in the image 
being processed appears to belong. The rules are 
then read out (Step 22) which define the relation- 
ship between the position coordinates of the pat- 
tern recognized among the R data and those of the 
pattern recognized among the G data as well as 
that between the position coordinates of the same 
pattern in the R data and those of the pattern 
recognized among the B data. This relationship 
determines whether these data represent the image 
which this pattern was determined to be. 

For the specified pattern to be detected shown 
in Figure 26, these rules will be statements such 
as: "For a certain specified pattern to be detected 
(which is a bank note), if there is a pattern like 
Pattern (1) in the R data, there will be a pattern like 
Pattern (3) in the G data. This pattern will have a 
reference point which is distance D away from the 
reference point (i.e., the specified position) of Pat- 
tern (1)." "For a certain specified pattern to be 
detected (which is a bank note), if there is a pattern 
like Pattern (3) in the G data, there will be a pattern 
like Pattern (4) in the B data. This pattern will have 
a reference point which is distance D' away from 
the reference point, i.e., the specified position, of 
Pattern (3)." The rules in this example will be valid 
even in the case of a printing misregister, since 
both the distances are given in terms of a fixed 
relationship (for D, that between D1 and D2; for D', 
that between D3 and D4). 

Next, the reference points in the R, G and B 
data of all the effective candidates are read out of 
memory 65. A judgment is made, based on the 
rules read out in the step discussed above, as to 
whether the image data being processed are or are 
not those of the specified pattern to be detected. 
Specifically, the point-to-point distance is obtained 
from the reference positions associated with any 
two data sets, and a judgment is made as to 
whether this distance is one of the fixed distances 
mentioned above (Step 23). 

Figure 25 shows the flow of judgment process- 
ing for the judgment (Step 23) when the two rules 
given above are used. First, the reference points (- 
(Xri, y,i) and (Xgj, y Bj ), where i = 0, 1, and j = 0, 



1, ...) of the extracted patterns which were recog- 
nized in the R and G data are read out. Based on 
these reference points, distances are obtained for 
every combination of points. A judgment is then 
5 made as to whether these data represent the speci- 
fied pattern to be detected by determining whether 
these distances fulfill the formula given below 
(Steps 23a through 23d). 

w Di i[(x ai - x ri ) 2 + (y gj - y ri ) 2 ]* ^D 2 

If the condition given above is met, the refer- 
ence points ((x b k, Ybk). where k = 0, 1, ...) of the 
extracted pattern which was recognized in the B 

75 data are read out. Based on these reference points, 
distances are obtained for every combination of 
points. A judgment is then made as to whether 
these data represent the specified pattern by deter- 
mining whether these distances fulfill the criterion 

20 given below (Steps 23e through 23g). 

Da 2[(Xbk - x fli ) 2 + (y bk - y Bj ) 2 ]* SD 4 

If the condition given above is also met, the 
25 judgment is made that the pattern, i.e., the speci- 
fied pattern to be detected, has been found, and a 
specified signal is output (Steps 23f and 24). Alter- 
natively, it would be possible to define distances D 
and D' as fuzzy values and perform the judgment 
30 discussed above as a fuzzy inference. 

In the second embodiment, even though the R, 
G and B data are read in field-sequential fashion, 
specified data (recognized patterns) are extracted 
after each set of image data is sent. Because the 
35 data stored in memory 65 are coded data concern- 
ing the coordinates of reference points of recog- 
nized patterns, only a small memory capacity is 
needed. Since inferences are performed by pat- 
tern-matching during the readout of the R, G and B 
40 image data (in other words, there is no need to wait 
for all the image data to be read out), the final 
judgment as to whether this is indeed the specified 
pattern to be detected can be made quickly. In this 
example, RGB data were used; however, the inven- 
ts tion is equally applicable to CMY data or to various 
other types of data. 

Figures 27 and 28 (A)-28(B) show the compo- 
nents of a third embodiment of an image process- 
ing device designed according to this invention. 
so The third embodiment represents an improvement 
of pattern matching device 3 in the basic configura- 
tion shown in Figure 1 and, more specifically, in 
the judging scheme used in CPU 30. The matching 
operation to detect a given image described above 
55 involved choosing as a reference pattern one which 
most clearly expressed a characteristic and then 
judging whether a pattern which was found con- 
tained an image resembling that reference pattern. 
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In this embodiment, when a single set of image 
data are judged, a number of previously selected 
reference patterns in various locations are used, 
and an overall judgment is made, using OR or AND 
conditions, as to the degree of resemblance of the 
image data with respect to each of the reference 
patterns. 

The flow of processing performed by CPU 30 
in Figure 10 is shown in Figure 27. The two speci- 
fied patterns shown in Figure 28(A), Ao and B 0 , are 
used to generate the characteristic count data 
which will specify the image to be detected. These 
patterns are stored ahead of time as reference 
patterns Ao and B 0 , which are shown in Figures 28- 
(B) and 28(C). 

The potential matches, i.e., patterns similar to 
the specified pattern to be detected, which were 
extracted by extraction device 2 are obtained, and 
each one is compared with the reference pattern 
by pattern matching device 3, which then deter- 
mines its degree of resemblance to the target. If 
one of the potential matches has a degree of 
resemblance to the reference pattern which ex- 
ceeds a specified reference value, that candidate is 
assumed to contain the specified pattern to be 
detected, and a specified control signal or a signal 
indicating the degree of resemblance can be output 
(See Step 27). In this example, different threshold 
values (Th (Ao) and Th (Bq)) were used for the 
degree of resemblance to the reference patterns Ao 
and Bo. It would also be possible to use the same 
threshold value. 

Let us assume that the pattern being pro- 
cessed consists of a portion of pattern B' shown in 
Figure 29. Let us further assume that the degree of 
resemblance between pattern B' and reference pat- 
tern Bo is lower than it should be as a result of a 
printing error. Since the degree of resemblance of 
pattern B" is low even though it actually is the 
image to be detected, it might be missed if only 
reference pattern B were used to make a judgment, 
as in previous devices. Since this embodiment 
uses two reference patterns, and the degree of 
resemblance of the other pattern, A', to its refer- 
ence pattern Ao will be high, the image can be 
detected accurately. 

In this embodiment the image was judged to 
contain the specified pattern if a high degree of 
resemblance was found with respect to one of the 
reference patterns. However, the invention is not 
limited to this case only. It would, for example, be 
possible to conclude that the image contained the 
specified pattern by using three or more reference 
patterns and finding that the degree of resem- 
blance to the reference pattern exceeded a speci- 
fied reference value for a number of these patterns 
(a value smaller than the total number of reference 
patterns). 



Another acceptable scheme would be to estab- 
lish at least two values, one high and one low, as 
references by which to judge the degree of resem- 
blance. If the degree of resemblance exceeded the 

5 low value with respect to all or a majority of the 
reference patterns, and if the degree of resem- 
blance exceeded the high value with respect to at 
least one of the reference patterns, it could be 
concluded that the specified pattern to be detected 

id had been found. 

In this third embodiment, which was discussed 
above, the judgment of whether or not the image 
contains the specified pattern, i.e., the one we hope 
to detect, is made by judging whether or not it 

is contains a pattern whose degree of resemblance to 
a certain reference pattern exceeds a specified 
value. Another embodiment might base the first 
judgment not only on the degree of resemblance to 
a number of reference patterns, but also on the 

20 spatial relationship between the patterns which 
have been found. 

Let us assume that the image data we hope to 
detect are those shown in Figure 30. Reference 
patterns A and B are identical to those shown in 

25 Figures 28(B) and 28(C). In the embodiment dis- 
cussed above, both patterns C and D in Figure 31 
and patterns E and F in Figure 32 would show a 
high degree of resemblance to reference patterns 
A and B. Both the image shown in Figure 31 and 

30 that shown in Figure 32 would thus be presumed to 
be the image to be detected, since both contain 
the specified pattern. As a result, the processing 
performed in the copier would be prohibited. How- 
ever, both the image data shown in Figure 31 and 

35 that shown in Figure 32 differ from the image to be 
detected, and as such, they should be permitted to 
be copied. 

When a judgment is made using a number of 
reference patterns as a basis, and some patterns 

40 are detected whose degree of resemblance ex- 
ceeds a given value, we must obtain the orientation 
of each pattern we have detected as well as the 
distance between each pair. A final judgment can 
then be made by determining whether or not these 

45 values are within the same range as those of a 
reference pattern. 

More specifically, we obtain in advance the 
shapes of the reference patterns to be detected in 
the image data as well as the distance between 

so them (absolute value A - B) and their angles qA 
and qB with respect to parallel reference lines Z 
(which indicate their orientation), and we store 
these data. 

The sequence of steps in a judgment which 
55 uses orientation as a criterion is shown in Figure 
33. First, the degree of resemblance of each pat- 
tern extracted by device 2 is determined with re- 
spect to both the reference patterns A and B. 
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Judgments are made as to whether these values 
are greater than the threshold values ThAo and 
Th B [) (Steps 30 to 32). 

If the degree of resemblance of both patterns 
is greater than a given value, angles qC and qD are 
obtained for extracted patterns C and D, and the 
difference between them is also obtained (Steps 33 
to 35). A judgment is made as to whether this 
difference is approximately equal to the difference 
between the angles of the reference patterns. If it 
is, the image is judged to be the one being sought, 
and a specified signal is output (Steps 36 to 38). 

Since the difference between the angles of the 
patterns shown in Figure 31 will not be equal to the 
reference value, the image in Figure 31 will not be 
mistakenly presumed to be the one we are seek- 
ing. 

The sequence of steps in a judgment which 
uses the distance between patterns as a criterion is 
shown in Figure 34. First, the degree of resem- 
blance of each pattern extracted by device 2 is 
determined with respect to both the reference pat- 
terns, A and B. Judgments are made as to whether 
these values are greater than the threshold values 
Th A0 and Th B0 (Steps 40 to 42). 

If both values exceed their thresholds, the dis- 
tance between extracted patterns E and F is ob- 
tained (absolute value E - F) (Steps 43 to 45). A 
judgment is made as to whether the distance be- 
tween these patterns is approximately equal to the 
distance between the reference patterns. If it is, the 
image is judged to be the one being sought, and a 
specified signal is output (Steps 46 to 48). 

Since the distance between the patterns in 
Figure 32 will not be equal to the reference value, 
the image in Figure 32 will not be mistakenly 
presumed to be the one we are seeking. 

The question of which sort of data to use when 
a judgment is to be based on data concerning the 
spatial relationship of the patterns as well as their 
degree of resemblance to reference patterns, as 
discussed above, is left to the user. Either orienta- 
tion or distance may be used, or both may be used 
together, or yet a third criterion may be added to 
the judgment processing. The relevant processing 
is performed by CPU 30, which is pictured in 
Figure 1 0. 

The embodiments discussed above were all 
devices suited for use in copy machines. The in- 
vention, however, is not limited to this application 
only. It is also suitable for use in color printers, fax 
machines, communication transmission devices, 
and various other types of equipment. 

The image processing device of this invention 
and the copy machine in which it is used have a 
lower cost than their predecessors because they 
require only a small-capacity memory. This im- 
provement is a result of the fact that the specified 



patterns which are detected constitute only a por- 
tion of the entire image. Thus these patterns can 
be detected regardless of the size (i.e., the exterior 
dimensions) of the document which may not be 

5 copied (or printed). Detection is based on whether 
or not these specified patterns can be found. All 
processing from detection of possible specified 
patterns in the first scan to determining their de- 
gree of resemblance to a reference pattern can be 

10 performed in real time. The specified pattern will 
not be missed even in a single-scan copy machine, 
and the recognition rate will be high. 

If for every search the received image is gra- 
dated and potential matches are compared to the 

75 gradated image (or pattern), the image data with 
the specified pattern to be detected can be ex- 
tracted quickly and accurately. If after the patterns 
are detected using the gradated image, one of 
them is selected to be output based on the spatial 

20 relationships among them, the result of using the 
gradated image for matching will be that spurious 
data, i.e., data which are not associated with a 
specified pattern to be detected, can be discarded. 
Even though extraneous data are collected, they 

zs can be barred from subsequent processing, such 
as being extracted or evaluated for resemblance to 
a reference pattern. This allows us to reduce the 
memory capacity and increase the processing 
speed. 

so Because the extraction device has the ability to 
detect certain specified positions, such as the apex 
of a corner or the center of a circle, in the pattern 
which has been detected, these positions can be 
specified accurately in the first stage of processing. 

35 This will allow a virtually perfect registration of the 
detected pattern with the reference pattern when a 
degree of resemblance is to be determined. If a 
specified position cannot be detected, the judg- 
ment can be made that the image being processed 

40 cannot be the specified pattern to be detected. The 
degree of resemblance then need not be calcu- 
lated, resulting in a further increase in processing 
speed. 

Because the device which calculates the de- 
45 gree of resemblance performs fuzzy inferences us- 
ing a number of items of fuzzy knowledge created 
from one of the reference patterns established in 
advance, erroneous judgments resulting from the 
image being slightly shifted in printing can be 
so effectively avoided with no loss in processing 
speed. 

Because the device which detects the pattern, 
the device which extracts the pattern and the de- 
vice which calculates its degree of resemblance to 
55 a reference all process the data as soon as a set of 
data for a color component arrives in field-sequen- 
tial fashion; and because a judgment is based on 
the spatial relationship of the potential matches 
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detected in these data, then the patterns can be 
matched as soon as a data set is received, even 
though the data are analyzed into RGB, CMY, or 
some other set of components. Since the spatial 
relationship of the patterns to each other is a 
criterion for a high degree of resemblance, there is 
no need to wait for the later fields of the image to 
be read out so that all the color data are sent. The 
various types of processing needed to make judg- 
ments are performed in parallel with the readout of 
data for each color. As a result, judgments can be 
produced at high speed and a small-capacity mem- 
ory can be used. 

Because reference patterns are provided which 
correspond to a number of patterns in the image 
data for a single image, and because judgments 
are based on all of these reference patterns, the 
desired image can be detected accurately even 
when the degree of resemblance of a portion cor- 
responding to one reference pattern is low as the 
result of unevenness or alteration of the print. This 
can be done by comparing another detected pat- 
tern to its reference pattern. The spatial relation- 
ships among a number of reference patterns are 
used as a criterion for a judgment. This insures that 
an image containing a pattern which is identical to 
a reference pattern but is located in a different part 
of the image will not be mistakenly identified as the 
image being searched for. 

By installing this image processing device in a 
copy machine, we can absolutely prevent the out- 
put of copies of documents whose reproduction is 
illegal, such as bank notes and negotiable securi- 
ties. 

Claims 

1. An image processing device for identifying an 
input image, the image processing device 
comprising: 40 

forming means for forming a gradated im- 
age from the input image; 

determining means for determining wheth- 
er the gradated image includes a specified 
pattern; 45 

extracting means for, when the determin- 
ing means determines that the gradated image 
includes the specified pattern, extracting a por- 
tion of the input image corresponding to the 
specified pattern; and so 

comparing means for comparing the ex- 
tracted portion of the input image with a refer- 
ence pattern to determine a degree of resem- 
blance between the extracted portion of the 
input image and the reference pattern. 55 

2. The image processing device according to 
claim 1, wherein the determining means in- 



cludes searching means for searching the gra- 
dated image for the specified pattern. 



5. The image processing device according to any 
of claims 1 to 4, 

wherein the comparing means generates a 
control signal in accordance with the degree of 
resemblance between the extracted portion of 
the input image and the reference pattern. 

6. The image processing device according to 
claim 5, wherein the comparing means in- 
cludes a fuzzy inference unit and a memory 
having fuzzy knowledge stored therein. 

7. The image processing device according to any 
of claims 1 to 6, 

wherein: 

the determining means determines wheth- 
er the gradated image includes a plurality of 
specified patterns; 

the extracting means, when the determin- 
ing means determines that the gradated image 
includes the plurality of specified patterns, ex- 
tracts portions of the input image correspond- 
ing to the plurality of specified patterns; and 

the comparing means compares the ex- 



3, The image processing device according to 
5 claim 2, wherein: 

the searching means specifies an approxi- 
mate reference position and orientation of the 
specified pattern; 

the extracting means extracts a first por- 
io tion of the input image corresponding to the 

specified pattern in accordance with the ap- 
proximate reference position and orientation, 
specifies a precise reference position and ori- 
entation of the specified pattern from the ex- 
15 traded portion of the input image, and extracts 

a second portion of the input image in accor- 
dance with the precise reference position and 
orientation, the second extracted portion of the 
input image being included within the first ex- 
20 tracted portion of the input image; and 

the comparing means compares the sec- 
ond extracted portion of the input image with 
the reference pattern to determine a degree of 
resemblance between the second extracted 
25 portion of the input image and the reference 

pattern. 

4. The image processing device according to any 
of claims 1 to 3, 

ao wherein the comparing means compares the 
extracted portion of the input image with the 
reference pattern using a fuzzy pattern match- 
ing logic. 
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determining means for determining wheth- 
er the gradated image includes a specified 
pattern, 

extracting means for, when the determin- 
5 ing means determines that the gradated image 

includes the specified pattern, extracting a por- 
tion of the input image corresponding to the 
specified pattern, and 

comparing means for comparing the ex- 
10 traded portion of the input image with a refer- 

ence pattern to determine a degree of resem- 
blance between the extracted portion of the 
input image and the reference pattern; and 
printing means for printing a copy of the 
75 input image from the color component data 

produced by the color processing circuit. 

13. The copier according to claim 12, 

wherein the comparing means generates a 
20 control signal in accordance with the degree of 

resemblance between the extracted portion of 
the input image and the reference pattern, the 
control signal preventing the printing means 
from printing the copy of the input image. 



tracted portions of the input image with a plu- 
rality of reference patterns to determine a de- 
gree of resemblance between the extracted 
portions of the input image and the plurality of 
reference patterns. 

8. The image processing device according to any 
of claims 1 to 6, 

wherein the input image comprises a plurality 
of color components and wherein: 

the forming means forms a gradated im- 
age for each of the color components; 

the determining means determines wheth- 
er each of the gradated images includes a 
specified pattern; 

the extracting means, when the determin- 
ing means determines that the gradated im- 
ages include the specified patterns, extracts a 
portion of the input image corresponding to the 
specified patterns for each of the color compo- 
nents; and 

the comparing means compares the ex- 
tracted portions of the input image with refer- 
ence patterns to determine degrees of resem- 
blance between the extracted portions of the 25 
input image and the reference patterns. 

9. The image forming apparatus according to 
claim 8, wherein the forming means, the deter- 
mining means, the extracting means, and the 30 
comparing means each operates using a sur- 
face sequence scheme for each of the color 
components. 

10. The image processing device according to any 35 
of claims 1 to 6, 

wherein the comparing means compares the 
extracted portion of the input image with a 
plurality of reference patterns to determine a 
degree of resemblance between the extracted 40 
portion of the input image and the plurality of 
reference patterns. 

11. The image processing device according to 
claim 10, further comprising means for deter- 45 
mining a spatial relationship between the plu- 
rality of reference patterns. 

12. A copier comprising: 

scanning means for scanning an original so 
image to produce an input image; 

a signal processing unit including a color 
processing means for producing color compo- 
nent data from the input image and an image 
processing device, the image processing de- 
vice including: 

forming means for forming a gradated im- 
age from the input image, 



14. The copier according to claim 12, 

wherein the comparing means generates a 
control signal in accordance with the degree of 
resemblance between the extracted portion of 
the input image and the reference pattern, the 
control signal causing the color processing 
means to execute a copy prevention routine. 

15. A method for identifying an input image, the 
method comprising the steps of: 

forming a gradated image from the input 
image; 

determining whether the gradated image 
includes a specified pattern; 

extracting a portion of the input image 
corresponding to the specified pattern when 
the gradated image includes the specified pat- 
tern; and 

comparing the extracted portion of the in- 
put image with a reference pattern to deter- 
mine a degree of resemblance between the 
extracted portion of the input image and the 
reference pattern. 

16. The method according to claim 15, 

wherein the determining step includes the step 
of searching the gradated image for the speci- 
fied pattern. 



55 17. The method according to claim 16, 
wherein: 

the searching step includes the step of 
specifying an approximate reference position 

17 
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and orientation of the specified pattern; 

the extracting step includes the steps of 
extracting a first portion of the input image 
corresponding to the specified pattern in ac- 
cordance with the approximate reference posN 5 
tion and orientation, specifying a precise refer- 
ence position and orientation of the specified 
pattern from the extracted portion of the input 
image, and extracting a second portion of the 
input image in accordance with the precise ;o 
reference position and orientation, the second 
extracted portion of the input image being in- 
cluded within the first extracted portion of the 
input image; and 

the comparing step includes the step of is 
comparing the second extracted portion of the 
input image with the reference pattern to deter- 
mine a degree of resemblance between the 
second extracted portion of the input image 
and the reference pattern. 20 

18. The method according to any of claims 15 to 

17, wherein the comparing step includes the 
step of comparing the extracted portion of the 
input image with the reference pattern using a 25 
fuzzy pattern matching logic. 

19. The method according to any of claims 15 to 

18, wherein the comparing step includes the 
step of generating a control signal in accor- 30 
dance with the degree of resemblance be- 
tween the extracted portion of the input image 

and the reference pattern. 

20. The method according to any of claims 15 to 35 

19, wherein: 

the determining step includes the step of 
determining whether the gradated image in- 
cludes a plurality of specified patterns; 

the extracting step includes the step of 40 
extracting portions of the input image corre- 
sponding to the plurality of specified patterns 
when the gradated image includes the plurality 
of specified patterns; and 

the comparing step includes the step of 45 
comparing the extracted portions of the input 
image with a plurality of reference patterns to 
determine a degree of resemblance between 
the extracted portions of the input image and 
the plurality of reference patterns. so 

21. The image method according to any of claims 
15 to 19, wherein the input image comprises a 
plurality of color components and wherein: 

the forming step includes the step of for- 55 
ming a gradated image for each of the color 
components; 

the determining step includes the step of 



determining whether each of the gradated im- 
ages includes a specified pattern; 

the extracting step includes the step of 
extracting a portion of the input image cor- 
responding to the specified patterns for each 
of the color components when the gradated 
images include the specified patterns; and 

the comparing step includes the step of 
comparing the extracted portions of the input 
image with reference patterns to determine 
degrees of resemblance between the extracted 
portions of the input image and the reference 
patterns. 

22. The method according to claim 21, 

wherein the forming step, the determining step, 
the extracting step, and the comparing step 
each includes the step of using a surface se- 
quence scheme for each of the color compo- 
nents. 

23. The method according to any of claims 15 to 
22. 

wherein the comparing step includes the step 
of comparing the extracted portion of the input 
image with a plurality of reference patterns to 
determine a degree of resemblance between 
the extracted portion of the input image and 
the plurality of reference patterns. 

24. The method according to claim 23, 

further comprising the step of determining a 
spatial relationship between the plurality of ref- 
erence patterns. 
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